Portable terminal device, program for reading information, and recording medium having the same recorded thereon

ABSTRACT

A portable terminal device and a program for reading information, which can read a code correctly with simple operations, and a recording medium having the program recorded thereon. A portable telephone with a camera has a camera for inputting information of an image of interest, a small light, which emits light through the operation of ON/OFF button while reading a bar code, an LCD display part for displaying information, and a control part including a CPU for controlling the whole device. The control part switches photography modes depending on the existence of objects such as natural images or bar codes. The control part recognizes a bar code when the device switches to a mode for bar code photography. Then, the device notifies a user of the results of the recognition and then performs a predetermined process based on the content of the recognition.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a Continuation of application Ser. No. 10/622,587filed on Jul. 21, 2003, now U.S. Pat. No. 7,090,136, and for whichpriority is claimed under 35 U.S.C. §120; and this application claimspriority of Application No. 2002-214733 filed in Japan on Jul. 24, 2002under 35 U.S.C. §119; the entire contents of all are hereby incorporatedby reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a portable terminal device such as aportable telephone with a digital camera or a camera, a program forreading information, and a recording medium having the program recordedthereon.

2. Background Art

Current portable telephones, which are provided with display devicessuch as a liquid crystal panel and input devices through key inputting,etc., tend to have multiple functions such as e-mail exchange andgame-playing, in addition to telephone functions for transmitting andreceiving radio waves.

In recent years, small image sensors with low power consumption havebeen developed, and this has allowed a camera to be built in a portableterminal device such as a portable telephone. Thus, it is possible tosend images by e-mail immediately after they are taken by a built-incamera. As for these cameras, priority is given to the fact that theyare small, and their resolution is lower than that of generallyavailable digital cameras.

Further, in recent years, portable telephones can be connected to theInternet. For the connection to the Internet, it is necessary to input aURL (Uniform Resource Locator) through the keys and it is troublesome toinput a long URL through the keys of a portable telephone. To solve thisproblem, a unique number is assigned to a URL, a bar code correspondingto the number is printed, and the printed bar code is read by a bar codereader, which avoids excessive user labor for inputting the URL. Thisattempt has been made to improve the usability for users. In the case ofsuch a device, an independent bar code reader is required to beconnected to a portable telephone.

However, in the case of a portable terminal device having a camera builttherein, when a bar code is recognized from a bar code image inputted bythe built-in camera, the above service is available without thepreparation of an independent bar code reader.

JP Patent Publication (Kokai) No. 2002-111909 A, for example, disclosesan information-providing service whereby a user can select and set amonga photograph mode and a QR code (two-dimensional code) reading mode, andwhere processing is conducted in accordance with the setting mode.Further, JP Patent Publication (Kokai) No. 9-259215 A (1997) discloses acode reader, which processes a one-dimensional code by a line CCD and atwo-dimensional code by an area CCD for enhancing reading accuracy.However, the setting of a camera image capture area in accordance withtypes of bar codes is not disclosed.

Furthermore, JP Patent Publication (Kokai) No. 2001-197186 A disclosesthat a telephone number is replaced with a two-dimensional bar code andthe two-dimensional bar code is read by a CCD scanner installed on atelephone.

In the meantime, the resolution of currently built-in cameras is lowerthan that of an image sensor used for a bar code reader and therefore ithas been difficult to recognize a bar code with high accuracy.

Further, the cause for the above difficulty in recognizing bar codes isthat the widths of bars photographed on an input image are not constant.This is because the distance between a bar code and a camera varieswhenever an image is inputted. In the case of a close-up type scanner, abar code can be always inputted in a constant size, and therefore thewidths of bars can be predetermined. However, when a bar code isphotographed by a hand held camera, it is impossible to predetermineconstant bar widths.

A method for recognizing a bar code by determining bar widths based on abar code image inputted by an image scanner has been proposed. Forexample, JP Patent Publication (Kokai) No. 2002-176509 A discloses adevice that recognizes a bar code included in an image captured by acamera, and generates speaker sound when the captured image isdecodable. At the same time, it saves the bar code data and displays itas an image on a display part.

However, conventional portable terminals with cameras have the followingproblems.

For example, since a one-dimensional code or a two-dimensional codediffers in physical size from a code to be read, suitable resolution hasto be selected depending on the size of codes in the case of a portabletelephone camera, which has a comparatively low resolution.

Also, a camera usually set for photographing persons or landscapes isnot anticipated for use in macro-mode photographing of a bar code, andthus a bar code image suitable for reading cannot be taken.

Furthermore, since a camera of a portable terminal device has acomparatively narrow angle of view, it is difficult to set a code to beread in a range photographed by the camera. In this case, close-upphotographing is necessary, and this causes a shadow of the main body ofthe device itself or the like, which is likely to affect a capturedimage, so that a stable image cannot be obtained.

SUMMARY OF THE INVENTION

The present invention has been achieved in view of the above problems,and an object of the present invention is to provide a portable terminaldevice that can accurately read a code with easy operation, a programfor reading information, and a recording medium having the programrecorded thereon.

Another object of the present invention is to provide a portableterminal device capable of easily setting a camera opening to correspondwith an object.

A portable terminal device of the present invention comprises:

-   -   image pick-up means for picking up an image;

photography procedure switching means for switching to a bar code as anobject to be photographed;

display means for displaying the image obtained by the image pick-upmeans; and

code recognition means for recognizing the picked-up bar code image whenthe device is switched to bar code photography mode by the photographyprocedure switching means.

It is more preferable that the photography procedure switching meanschanges an image capture area in the image pick-up means, depending onthe type of bar code.

It is more preferable that the photography procedure switching meanschanges at least one or more of the factors of gradation property,contrast, brightness, white balance, exposure, focus depth, edgeenhancement, and resolution between a natural image and a bar codeimage.

It is preferable that the photography procedure switching means hasframe rate control means, which, when switching to bar code photographymode, displays an image obtained by the image pick-up means at a highframe rate on the display means until the code recognition means startsup and displays an image at a frame rate necessary for the coderecognition means after the code recognition means starts up.

Further, the portable terminal device of the present inventionpreferably has lighting means for lighting the object, and furthermore,it may have lighting switching means for turning on/off the lightingmeans.

In addition, the portable terminal device preferably has notifying meansfor giving notice of a result recognized by the code recognition means.

The notifying means gives notice of the recognized result via sound,voice, or a message displayed on the display means.

The portable terminal device also has timer setting means for measuringa setting time, wherein the notifying means gives notice of recognitionfailure when a bar code is not recognized within a period set by thetime setting means.

A portable terminal device of the present invention comprises a camerafor picking up an image and a display part for displaying the picked-upimage, both being built therein, wherein a marker indicating cameraposition is provided.

The marker is provided on the surface of a main body just at the back ofan opening of the camera by a printing, an inscription, or a sticker.

Further, when the display part is located on the main body just at theback of the opening of the camera, the portable terminal device may havemarker display means for displaying image information indicating themarker on the display part.

The marker display means preferably deletes the display of the imageinformation indicating the marker in synchronization with the shutteroperation of the camera or at the same time of a notice of image pick-upresult, or switches on/off the display of the image informationindicating the marker in accordance with user's instruction.

Further, according to a more preferable embodiment, the image pick-upmeans picks up a moving/still image with an area-type solid-state imagesensing device.

According to a more preferable embodiment, the portable terminal deviceis a portable telephone or portable information terminal.

Further, the present invention provides a computer-readable recordingmedium having a program recorded thereon, the program causing a computerto operate a portable terminal device comprising:

-   -   image pick-up means for picking up an image;

photography procedure switching means for switching to a bar code as anobject to be photographed;

display means for displaying the image obtained by the image pick-upmeans; and

code recognition means for recognizing the picked-up image of the barcode when the device is switched to bar code photography mode by thephotography procedure switching means.

Furthermore, the present invention provides a program causing a computerto operate a portable terminal device comprising:

image pick-up means for picking up an image;

photography procedure switching means for switching to a bar code as anobject to be photographed;

display means for displaying the image obtained by the image pick-upmeans; and

code recognition means that recognizes the picked-up image of the barcode when the device is switched to bar code photography mode by thephotography procedure switching means.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a view showing the outer appearance of a portable terminaldevice according to an embodiment of the present invention.

FIG. 2 is a block diagram showing a case wherein a bar code reader ofthe portable terminal device of this embodiment is incorporated into apotable telephone with a camera.

FIG. 3 is a schematic view showing a mechanism for switching the depthof camera lens focus of the portable terminal device according to thisembodiment.

FIG. 4 is a flow chart showing bar code reading operations of theportable terminal device according to this embodiment.

FIG. 5 is a view showing an example of mode selection screen display onan LCD display part of the portable terminal device according to thisembodiment.

FIG. 6 is a view showing that image capture areas and magnifications atthe time of displaying according to the portable terminal device of thisembodiment has different are varied, depending on the one-dimensional ortwo-dimensional code.

FIG. 7 is a view showing screen display examples obtained by the barcode reading process of the portable terminal device of this embodiment.

FIG. 8 is a view showing screen display examples obtained by the barcode reading process of the portable terminal device of this embodiment.

FIG. 9 is a view showing the outer appearance of a portable terminaldevice according to another embodiment.

FIG. 10 is a view showing the outer appearance of a portable terminaldevice according to another embodiment.

FIG. 11 is a flow chart showing bar code reading operations of theportable terminal device according to another embodiment.

DETAILED DESCRIPTION OF THE INVENTION

This application is based on the Japanese patent application2002-214733, filed on Jul. 24, 2002, all the contents of which isincorporated in this application by reference.

Hereinafter, preferred embodiments of portable terminal devicesaccording to the present invention will be described in detail byreferring to the accompanying figures.

First, the basic concept of the present invention is described. A casewherein a portable terminal device of the present invention is appliedto a portable telephone with a camera is taken as an example.

(1) The portable telephone with a camera has a small light builttherein, which can be turned on or off through the user's operation.When the camera reads a two-dimensional bar code, for example, the lightis turned on or off through the user's operation. Even when theperipheral light quantity is reduced due to lens properties or a shadowof the main body of the telephone overlaps the code to be read, the codecan be stably read by the user turning on or off the light. In anembodiment wherein the light is turned on whenever reading codes,adverse effects may be caused by specular reflection. Thus, the light isswitched on or off while checking the image being inputted with thecamera on an LCD display part.

(2) The portable telephone with a camera allows an image of code beingpicked up to be displayed on an LCD display part in real time whenreading the code. A user can confirm that the bar code has beencorrectly photographed by a camera through the display of the image ofthe code being picked up in real time. This prevents the code fromprotruding beyond the screen, as well as the generation of a largelydeformed image thereof, thereby improving the operability. Further,after the reading, the results of the reading are immediately displayedon the screen, whereby they can be instantly known. Furthermore, whenthe reading is completed, a reading completion sound such as a “ping” or“tinkle” may be used to inform a user of the completion of the reading.

(3) Reading operations are performed continuously on each image capturedsuccessively, and continued until the reading is completed. At thistime, it takes a long time for reading process, so the frame rate ofcapturing images is at most approximately 2FPS (frame per second) andthat of displaying is also approximately 2FPS. This condition makes itdifficult to set a code to a suitable position while watching the LCDdisplay part. Therefore, the portable telephone with a camera accordingto the present invention does not perform the reading process on a codeuntil a user provides instructions, so that smooth moving images aredisplayed on the LCD display part. This allows the user to suitablyposition a code. After the code is placed in a suitable position, abutton is operated to permit the device to perform the reading process.At this time the frame rate of displaying decreases to about 2 FPS, butthe code has already been photographed at the suitable position, andthus the reading can soon be finished without a problem.

(4) Moreover, in the case of an ordinary portable telephone with acamera, color shade or the like is absolutely adjusted to be natural forthe purpose of photographing persons and landscapes. Due to this, it isnot suitable for use in reading codes. The portable telephone with acamera according to the present invention has a mode that is preparedand adjusted for code reading, which is different from ordinary modes.When reading codes, this mode is used. For example, in order toemphasize the bright and dark contrast of a code, white balance orexposure is adjusted, correction factors are adjusted for emphasizingborders (edge portion), or a special mode for bar code reading havingadjusted resolution is prepared. When a user designates code reading,the telephone is automatically switched to the above bar code readingmode.

(5) The code to be read is classified into types such as one-dimensionalcode or two-dimensional code. Depending on the types of these codes, thesize or line thickness of printed code is different. Thus, the presentportable telephone with a camera allows a user to designate which codeis to be read (one-dimensional code or two-dimensional code), and inresponse to user's designation, the image capture area of the camera ischanged. Also, the magnification is optimized at the time of displaying.

FIG. 1 is a view showing the outer appearance of a portable terminaldevice according to this embodiment of the present invention based onits basic concept. FIG. 1 (A) is a front view thereof, and FIG. 1 (B) isa rear view of thereof.

The portable terminal device of this embodiment is an example wherein acode reader is applied to a portable communication terminal such asportable telephone or PHS (Personal Handy-Phone System) with a camera.

FIG. 1 shows a portable telephone 10 with a camera (portable terminaldevice). A portable telephone main body 10 a comprises an antenna 11 forconducting wireless communications, a camera 12 (image pick-up means)for inputting image information of interest, the camera being built-inthe rear face of the main body 10 a, a small light 13 (lighting means),built-in the rear face of the main body 10 a, which is lit throughuser's operations, etc. when reading bar codes or being used in a darkplace, a speaker 14 (a part of notifying means) for receiving voices andamplifying them, an LCD display part 15 (a part of the display means andnotifying means) for displaying information such as guidance foroperations, received information, images, or text information, a powerkey 16 for turning the telephone on or off, a telephone function key 17for conducting, for example, the reception, transmission, switching overtelephone functions, determination of operations, a mode key 18 forswitching over various functions, cursor keys 19 for moving a cursorfrom right to left or up and down to select an object of interest, ashutter key 20 that works as a shutter button to determine the input ofthe camera 12, a light ON/OFF button 21 for turning the light 13(lightning switching means) on or off, dial keys 22 including membranekeys for dialing such as inputting telephone numbers, and a microphone23 for inputting voices.

The camera 12 is a CCD (Charge Coupled Device: an area-type solid-stateimage sensing device) camera, which is built into the portable telephone10 and has, for example, 340,000 pixels. The camera 12 has a unifocallens and can change the depth of focus between the time of ordinaryphotography and the time of reading bar codes. Detailed configurationwill be described by referring to FIG. 3.

The LCD display part 15 comprises an LCD display having a dot matrixstructure, a PDP as a backlight, an EL, and each driver. Imageinformation to be inputted through the camera 12 is displayed on the LCDdisplay 15 while photographing. When code recognition is completed, theresults of the recognition will be displayed on the LCD display withmessages such as characters or icons. Further, while not photographing,numbers inputted through the dial key 22, various screens fornon-telephone functions, icons indicating operation states of telephonefunctions, etc. may be displayed. Furthermore, the time is displayedduring the standby period, and, in addition, personal information, thestate of system or the like is displayed.

The power key 16, telephone function key 17, mode key 18, cursor keys19, shutter key 20, light on/off button 21 and dial key 22 are allmanually operated, and the contents of the operations as information aretransmitted to the control part 31 (FIG. 2). According to thisembodiment, the shutter key 20 or light on/off button 21 is independent,but the functions of these keys may be assigned to other keys (forexample, mode key 18).

FIG. 2 is a block diagram showing a case wherein a bar code reader isincorporated into a portable telephone with a camera.

In FIG. 2, the portable telephone 10 is a portable information terminalsuch as a PDA (Personal Digital Assistant), or a PHS or portabletelephone. The portable telephone 10 is configured to have a controlpart 31 including a CPU for controlling the entire device (photographyprocedure switching means, code recognition means, informationprocessing means, frame rate control means, and timer setting means); astorage memory 32, which comprises a semiconductor memory such as a ROMfor storing a control program, fixed data or the like, a RAM as astorage region for CPU operations, and a hard disk, and which storestext information, image information, and voice signals, those beingreceived or inputted; a camera 33 (image pick-up means) for inputtingimage information of interest; a light 34 (lighting means) for emittinglight through use's operation while reading bar codes; a display part 35(a part of the display means and notifying means) for displaying textinformation or image information to a user; a key input part 36 forinputting and generating key operation information from a user; anantenna 37 for transmitting and receiving radio waves, a wireless part38 for controlling the transmission and reception of wirelesscommunications; and a speaker 39 (a part of the notifying means) foroutputting voice signals, and a microphone 40 for inputting voicesignals from a user.

The control part 31 controls the whole portable telephone 10 and at thesame time has a bar code reader. Therefore, based on key operations, thecontrol part 31 controls the steps of photographing bar codes with thecamera 33, displaying image information, and recognizing bar codes, andthen performs the process based on the content of the recognition.

Particularly, the control part 31 functions as photography procedureswitching means for switching photography procedures depending theexistence on an object such as a natural image or a bar code, coderecognition means for recognizing (decoding) bar codes when the deviceis switched to the bar code photography mode, notifying means for givingnotice of the results of the recognition to a user, and informationprocessing means for conducting a predetermined processing based on thecontent of the recognition.

Further, when the device is switched to bar code photography mode,images obtained by the camera 33 are displayed on the display part 35 ata high frame rate until the code recognition means starts up. After thestart-up of the code recognition means, images are displayed at a framerate required for the code recognition means. In this way, the controlpart 31 also has a function as frame rate control means. The detaileddescription of the above individual functions will be made using theflow chart of FIG. 4.

The ROM is a read-only semiconductor memory that stores fixed data suchas communications control data and programs necessary for the operationsof the control part 31. The RAM is used as a working memory, whichtemporarily stores data concerning bar code recognition orcommunications, data used for calculation, the results of thecalculation, etc. The program processed by the portable telephone 10 isdeveloped and executed in this RAM. Also, a part of RAM comprises anEEPROM (electrically erasable programmable ROM), which is a non-volatilememory and electrically rewritable. Specifically, various specificationsin the portable telephone 10 can be modified by changing a program sothat the program can be recorded on the EEPROM. In other words,non-volatile memories such as EPROM and EEPROM have recently been usedfor a program ROM in order to avoid time loss from changing a mask ROMwhenever debugging for system development, thereby resulting in largereduction of the period for program development and modification.Further, if the program is downloaded for rewriting the program contentof EEPROM, it is possible to easily upgrade or modify the functions.

FIG. 3 is a schematic view showing a mechanism for switching the depthof lens focus of the camera 12, and shows a depth-of-focus switchingmechanism at the time of ordinary photographing and at the time ofreading bar codes.

In FIG. 3, the mechanism is configured to have a camera module 50including a solid-state image sensing device such as CCD and a signaloutput circuit thereof, a lens unit 52 having a lens 51 therein andbeing mated by screws with the top of the camera module 50, and a cameralever 53 mounted on the outer region of the lens unit 52 and forrotating the lens unit 52.

The lens unit 52 is mated by screws with the camera module 50. When apinch 53 a of the camera lever 53 is moved in the direction of A←→B asshown in FIG. 3, the lens unit 52 moves in the direction of X←→Yrelative to the camera module 50. This can change the focal lengthbetween the lens 51 mounted on the lens unit 52 and the camera module50, and thereby the depth of focus is switched between ordinaryphotography and bar code reading. This embodiment has a configuration sothat, when reading bar codes, a driving part (not shown) comprising asolenoid, etc., which receives drive signals from the control part 31,allows the camera lever 53 to slide so as to automatically change thedepth of focus. However, the main body 11 a of camera may be providedwith a sliding pinch synchronized with the pinch 53 a of the cameralever 53, and a user can manually change the depth of focus at a time ofmacro photography for bar code reading, etc.

Hereinafter, the operations of the portable telephone with a camerahaving the above configuration will be described.

FIG. 4 is a flow chart showing the operations for reading a bar code bythe portable telephone 10 and the operations are conducted by thecontrol part 31. In the figure, “S” represents each step.

FIG. 5 is a view showing an example of mode selection screen display onthe LCD display part 15 of the portable telephone 10.

When a user presses a mode key 18 during stand-by state, a modeselection screen is displayed as shown in FIG. 5(A). In this state, “barcode” is selected with a cursor key 19 and the mode key 18 or atelephone function key 17 is pressed for the determination of mode.Then, the telephone is shifted to a bar code reading mode, and a codetype selection screen is displayed as shown in FIG. 5(B).

The flow of FIG. 4 starts through “bar code” selection operation of theuser. First, a bar code reading mode setting is conducted in step S1. Inthis bar code reading mode setting, initial settings necessary forreading a bar code are established. In this initial setting, processesexcept bar code reading process, such as camera-photographing andschedule management, are properly finished and the data therefor issaved. The program necessary for reading a bar code is then started, andthe switching mechanism for the depth of focus necessary for reading abar code is operated as shown in FIG. 3.

In step S2, the type of code is determined upon the receipt of user'sinstructions in response to the code type selection screen as shown inFIG. 5(B). When one-dimensional code is selected as the type of code,the routine advances to S3. When two-dimensional code is selected, theroutine advances to S4.

When the bar code reading mode is set, the type of code (one-dimensionalcode/two-dimensional code) is selected by the user. As described above,there are several types of codes to be read, such as one-dimensionalcodes and two-dimensional code. The sizes or the line widths of printedcodes vary depending on the type of code. Therefore, the user designateswhether a one-dimensional or a two-dimensional code is to be read, andin accordance with user's designation, the image capture area of thecamera and magnification ratio while displaying are optimized.

In the case of a one-dimensional code, a predetermined value (settingvalue 1) is set for the camera and display in step S3.

In the case of a two-dimensional code, a value different from the abovesetting value 1 (setting value 2) is set in step S4.

FIG. 6 is a view illustrating that image capture areas andmagnifications at the time of displaying are varied depending on theuser of one-dimensional or two-dimensional code.

As shown in FIG. 6, in the case of one-dimensional code, the areacaptured by the camera is set to be 640 (height)×480 (length), and theimage display is set to be equal to the image captured by the camera. Incontrast, in the case of two-dimensional code, the area captured by thecamera is set to be 320 (height)×240 (length), and the image display isset to be twice as large in height and length as the image captured bythe camera. Namely, in the case of one-dimensional code, the capturedarea is set to fit a bar code having a large amount of information in aone-dimensional direction into an area photographed horizontally by thecamera. In the case of two-dimensional code, the area photographed bycamera (camera-captured-area) is reduced by half vertically andhorizontally, and then this camera-captured image is magnified twicevertically and horizontally to be displayed. In addition to this, othersettings such as enhancement of edge components and contrast enhancementmay be commonly used as camera settings for both one- andtwo-dimensional codes.

Referring back to the flow chart of FIG. 4, when camera/display setting1 (S3) or camera/display setting 2 (S4) is performed, the device is in astandby condition for event (S5). Depending on individual events,processes are changed over.

The event for light ON/OFF operation occurs when a user presses thelight ON/OFF button 21. When this event is recognized in Step 5, theroutine advances to Step 6 to turn on or off the light 13 according tothe setting.

The event for reading start operation occurs when a user presses thereading start button (e.g., telephone function key 17). When this eventis recognized in Step 5, the routine advances to Step 7 to set a flagindicating reading start. Further, as described below in FIG. 10, whenthe device has a configuration such that a mark indicating the positionof the camera 12 is displayed on the LCD display part 15, the mark maynot be displayed after starting the reading.

The event for frame capture occurs when the camera 12 reads an image forone frame and then finishes the reading. When this event is recognizedin Step 5, the routine advances to Step 8 to display the captured imageon the LCD display part 15 in accordance with the setting. At this time,the mark 71 (see FIG. 10) indicating the position of the camera 12 maybe displayed simultaneously on the LCD display part 15.

In Step 9, it is judged whether or not to start the reading process.When the reading start flag is set up (YES in Step 9), the code readingprocess starts in Step 10. When the reading is aborted, the routinereturns to Step 5 for recognizing an event.

In Step 11, it is judged whether or not the code reading is successfullyachieved. When the code reading is successfully completed (YES in Step11), the captured result is displayed on the LCD display part 15 in Step12. Then, in Step 13, a confirmation sound is played back with a speaker14, and the code reading according to this flow chart is completed. Byplaying back the confirmation sound to inform a user of the completionof the reading, the user's attention is drawn to the completion, and theuser knows the bar code reading is completed.

Before the reading starts, the loop enclosed by the broken line in FIG.4 is repeated. During this period, the code reading process is notconducted, and thus smooth display at a high frame rate is available.

FIG. 7 is a view showing screen display examples obtained by the abovebar code reading process. FIG. 7(A) shows a photography screen beforethe reading process. FIG. 7(B) shows a photography screen during thereading process. FIG. 7(C) shows a screen of the recognized result afterthe reading process.

Although the description will be made using an example wherein a patternto be recognized is a two-dimensional code, images to be recognized maybe character patterns in addition to symbol patterns such asone-dimensional/two-dimensional bar codes.

Before the reading starts, smooth display at a high frame rate isavailable. Therefore, for example, as shown in FIG. 7(A), a user caneasily handle the camera 12 (main body 10 a of portable telephone) sothat a code is positioned within the screen while watching to make surethat the code does not extend beyond the screen (captured area). Inother words, since the delay in displaying images of the code on thescreen before the reading is very short, a user can operate the deviceso as to locate the code within the screen while watching the screennaturally. Then, as shown in FIG. 7(B), when the code is located withinthe screen, the user operates the device to conduct the reading andthereafter the result of the reading is displayed (see FIG. 7(C)).

FIG. 8 is a view showing screen display examples obtained by the abovebar code reading process. FIG. 8(A) shows a photography screen beforethe reading process. FIG. 8(B) shows a photography screen during thereading process. FIG. 8(C) shows a screen of the recognized result afterthe reading process.

Further, as shown in FIG. 8(A), when a part of the image is dark due toshade or the like, the code reading becomes difficult. The code readingis conducted by bringing the camera 12 (main body 10 a of portabletelephone) close to a code to be read for photography, and this islikely to cause shade.

In this embodiment, a user can confirm the state of currently inputtedimage by watching the screen. Thus, when the code is shaded as shown inFIG. 8(A), the light ON/OFF button 21 is pressed to turn on the light13. Then, the influence of the shade is removed as shown in FIG. 8(B),and the reading is successfully achieved (see FIG. 8(C)).

FIG. 9 is a view showing the outer appearance of a portable terminaldevice according to another embodiment. FIG. 9(A) is a front viewthereof, and FIG. 9(B) is a rear view thereof. In FIGS. 1 and 9, likeparts are designated by like reference numbers.

In FIG. 9, a portable telephone 60 with a camera (portable terminaldevice) has a mark 61 indicating the camera position on the back surfaceof the main body 10 a and just at the back of the camera 12 (oppositeposition of the opening of the camera 12) with a sticker, a printing orthe like.

Since a user cannot see the camera position while visually observing anLCD display part 15, it is rather difficult to move the camera 12 to anobject. Particularly, when the camera should be brought close to a barcode for bar code reading, the above problem is remarkable.

In this embodiment, a mark 61 is attached just at the back of theopening of the camera 12, and therefore this mark 61 becomes a kind oftarget for moving the camera 12 to a code to be read as an object, sothat the user can operate it easily.

Further, according to the position of the camera 12, the mark may bedisplayed on the LCD display part 15 (marker display means) as shown inFIG. 10.

In FIG. 10, a portable telephone 70 with a camera (portable terminaldevice) displays a mark 71 indicating the camera position on the LCDdisplay part 15 on the back surface of the main body 10 a and just atthe back of the camera 12 (opposite position of the opening of thecamera 12).

When the mark is displayed on the LCD display part 15, it is preferableto delete the mark when operating the shutter. Further, when the devicehas this configuration, the mark may not be displayed after the readingstarts in the event of reading start operation (S7) in FIG. 4. Further,when the captured image in Step 8 of FIG. 4 is displayed on the LCDdisplay part 15, the mark 71 indicating the position of the camera 12may be displayed simultaneously.

As described above, since the mark 71 is provided at a positioncorresponding to the position of the camera 12, the mark is used as atarget to bring the camera 12 over a code to be read as an object,thereby resulting in easy operation.

FIG. 11 shows an example as another embodiment of the present invention.FIG. 11 is a flow chart showing bar code reading operations according toanother embodiment. In FIGS. 4 and 11, like parts are designated by likereference numbers.

In FIG. 11, steps corresponding to Steps 7 and 9 of FIG. 4 are omitted.In this case, after frame capture event is recognized in Step 5, theimage is displayed in Step 8. Then, the routine immediately advances toStep 10, and the code reading process is conducted.

In this embodiment, an image cannot be displayed at a high frame ratebefore starting the code reading. However, user's operations such aspressing a reading start button (e.g., telephone function key 17), canbe omitted, and thus it is advantageous that the number of operationscan be decreased.

Further, in another embodiment, an image captured by a camera is storedin a storage memory and the image is displayed. Then, the process isstopped once. During the stoppage, a user may be allowed to determinewhether or not to start the code reading process using the displayedimage. When the user selects the code reading start, the code readingstarts. Otherwise, the stored image is discarded, and a new image iscaptured by the camera.

This embodiment is effective for a case wherein a portable terminal hasa remarkably low processing performance and takes one second or more forthe code reading process. A user can judge whether or not to start thecode reading process after observing the captured image thoroughly.Therefore, images that obviously cause reading failure (blurred imagesdue to hand shaking, etc.), are not used for the code reading, and theimage is captured again. This reduces the number of the code readingprocesses to be performed to the minimum.

As described above, the portable telephone 10 with a camera according tothis embodiment comprises a camera 12 for inputting image information ofinterest, a small light 13 for emitting light by operating an ON/OFFbutton 21 when reading a bar code, an LCD display part 15 for displayingthe information, and a control part 31 including a CPU, which controlsthe entire device. The control part 31 switches photography proceduresdepending on the existence of objects such as natural images or barcodes. When the device is switched to bar code photography, the controlpart recognizes a bar code and notifies a user of the recognized result,and conducts predetermined processes based on the recognized contents.Therefore, when a bar code is read by a camera of a portable telephone,which has a relatively low resolution, the camera automatically shiftsto a bar code reading mode suitable for photographing bar codes, andcodes can be read correctly with easy operations.

Further, in switching to bar code photography mode, images obtained bythe camera 33 are displayed on the display part 35 at a high frame rateuntil the code recognition process starts. After the code recognitionprocess starts, images are displayed at a frame rate necessary for thecode recognition process. Therefore, the reading operations arepreformed continuously on each image captured successively, and smoothmoving images can be displayed on the LCD display part 15, so that thecode can be positioned appropriately.

The above description illustrates preferred embodiments of the presentinvention, but the scope of the present invention is not limitedthereto.

Incidentally, the above embodiments show examples suitable for aportable telephone. However, the present invention is applicable toportable information terminals such as PDAs, and other informationprocessing devices such as PCs, as long as they have cameras(built-in/externally equipped) provided therewith. Further, a code to beread may be any code.

Further, in the above embodiments, the term “portable terminal device”is used, but this term is used for the description as a matter ofconvenience. Thus, needless to say, it may be a communication terminaldevice, an information reading device, or an information reading method.

Furthermore, in the above embodiment, messages displayed by charactersor icon on the display part 15, and reading completion sounds through aspeaker 14 are described as notifying means. However, any notifyingmethod can be used, such as voices. When voices are used fornotification, the portable telephone is provided with a voice outputpart comprising a voice data generation chip, a D/A converter, anamplifier, and a speaker. Then, voice synthesis containing messagecontent corresponding to reading completed/reading failure is performedusing the voice data generation chip, and then the synthesized voice isoutputted through the speaker. Further, messages indicating that theuser should turn on the light 13, or that it is necessary to change thetype of bar code, are displayed on the display part 15. In addition tothe displaying, voices or sounds can be used for notification.

Moreover, the type and the number of and connection methods amongindividual circuit parts, for example, storage parts or display parts,which constitute the above portable terminal device, are not limited tothe above embodiments.

Further, the portable terminal device described above can be achieved bya program that causes these portable terminal devices to function. Thisprogram is stored in a computer-readable recording medium. In thepresent invention, this recording medium may be the main memory of thecontrol part 31 as shown in FIG. 2. Alternatively, a device for readingthe program such as a CD-ROM drive is provided as an external storagedevice, and the recording medium may be other medium containing aprogram, such as a CD-ROM readable by inserting it into the device. Ineither case, the stored program may be accessed and executed by the CPUof the control part 31. Alternatively, in either case, the program isread out, the read-out program is downloaded in a program storage area(not shown), and then the program is executed. In this case, a programfor download is supposed to be stored in each device in advance.

Here, the above medium containing a program is a recording mediumconstituted to be separable from a portable terminal device or aninformation processing device, and it may be a medium carrying a programfixedly. Examples thereof include: disks, e.g., magnetic disks includingmagnetic tapes and cassette tapes, and optical disks including CD-ROMs,CD-R/RWs, MOs, MDs, DVD-ROMs, DVD-RAMs, and DVD-RWs; cards, e.g., PCcards, compact flash cards™, smart media™, IC cards, SD cards™, andmemory sticks™; and semiconductor memories, e.g., mask ROMs, EPROMs,EEPROMs, and flash ROMs.

Further, it may be a medium carrying a program flexibly, such as in thecase of downloading a program from a communications network throughcommunication connection means connectable to an Internet provider or anexternal communications network such as a server terminal 400. However,when the program is downloaded from a communications network asdescribed above, a program for download is stored in advance orinstalled from another recording medium. The content stored in therecording medium is not limited to a program, but may also be data.

As described above, according to the present invention, a portableterminal device such as a portable telephone with a camera enables codesto be read precisely with easy operations.

Further, the present invention can provides a portable terminal devicethat allows the opening of the camera to be easily positioned on anobject thereby to enhance the usability and the operability of codereading.

1. A portable terminal device comprising: an imaging unit that images anobject; an imaging procedure switching unit that switches an imagedobject to a code; a display unit that displays an image produced by saidimaging unit; a code recognizing unit that recognizes a code of saidimaged object; a control unit that causes, once the imaged object isswitched to a code by said imaging procedure switching unit, imageinformation to be displayed on said display unit with an increasedmagnification; wherein the code recognition is performed continuouslywhile said image is continuously displayed on said display unit at aframe rate required by said code recognizing unit.
 2. The portableterminal device according to claim 1, wherein said control unit changesthe display magnification of image information on said display unit inaccordance with the type of code.
 3. The portable terminal deviceaccording to claim 1, wherein said imaging procedure switching unitchanges the display magnification of image information on said displayunit.
 4. The portable terminal device according to claim 3, wherein saidimaging procedure switching unit changes an image captured area in saidimaging unit depending on the type of code.
 5. The portable terminaldevice according to claim 1, wherein said code recognizing unit iscapable of recognizing a one-dimensional code; and wherein an imagecaptured area and the display magnification are changed so that thelonger side of said one-dimensional code can be captured by said imagingunit in the lateral direction thereof.
 6. The portable terminal deviceaccording to claim 1, wherein said imaging procedure switching unitchanges at least one of gradation property, contrast, brightness, whitebalance, exposure, focus depth, edge enhancement, and resolution whenthe imaging unit is switched to the code mode.
 7. The portable terminaldevice according to claim 1, further comprising a notification unit fornotifying the result of recognition by said code recognizing unit. 8.The portable terminal device according to claim 7, wherein saidnotification unit notifies the result of recognition via sound, voice,or a message displayed on said display unit.
 9. The portable terminaldevice according to claim 7, further comprising a timer setting unitthat counts a set time, wherein said notification unit notifies thefailure of recognition upon failure to recognize a code within a timeset by said timer setting unit.
 10. The portable terminal deviceaccording to claim 1, wherein the result of recognition is displayed onsaid display unit after code recognition is completed.
 11. The portableterminal device according to claim 1, wherein said portable terminaldevice is a cellular phone or a portable information terminal.
 12. Theportable terminal device of claim 1 wherein said control unit sets afirst display magnification when a first type of code is selected andsets a second display magnification different than said first displaymagnification when a second type of code is selected.
 13. The portableterminal device of claim 1 wherein said control unit sets a firstdisplay magnification when a first type of code is detected and sets asecond display magnification different than said first displaymagnification when a second type of code is detected.
 14. A portableterminal device comprising: an imaging unit that images an object; adisplay unit that displays an image produced by said imaging unit; acode recognizing unit that recognizes a code of said image; a coderecognition instructing unit that activates said code recognizing unitin response to a user instruction; and an imaging procedure switchingunit that switches the imaging unit from an imaging mode to a code mode,wherein said imaging unit is disposed on a back side of a main body, andsaid display unit is disposed on a front side of said main body, whereinafter the imaging unit has been switched to the code mode by saidimaging procedure switching unit and before said code recognizing unitis activated, said image is displayed on said display unit continuouslywith the display magnification changed in accordance with the type ofcode that can be read, and wherein, after said code recognizing unit isactivated, code recognition is performed continuously while said imageis displayed on said display unit at a frame rate required by said coderecognizing unit, and, once code recognition is successfully completed,the result of code recognition is displayed on said display unit insteadof said image.
 15. The portable terminal device according to claim 14,wherein said imaging procedure switching unit changes an image capturedarea in said imaging unit depending on the type of code.
 16. Theportable terminal device according to claim 14, wherein said imagingprocedure switching unit changes display magnification of imageinformation on said display unit depending on the type of code.
 17. Theportable terminal device according to claim 14, wherein said type ofcode includes the difference between a one-dimensional code and atwo-dimensional code.
 18. The portable terminal device according toclaim 14, wherein said imaging procedure switching unit changes at leastone of gradation property, contrast, brightness, white balance,exposure, focus depth, edge enhancement, and resolution when the imagingunit is switched to the code mode.
 19. The portable terminal deviceaccording to claim 14, further comprising a notification unit fornotifying the result of recognition by said code recognizing unit. 20.The portable terminal device according to claim 19, further comprising atimer setting unit that counts a set time, wherein said notificationunit notifies the failure of recognition upon failure to recognize acode within a time set by said timer setting unit.
 21. The portableterminal device according to claim 14, wherein the result of recognitionis displayed on said display unit after code recognition is completed.22. The portable terminal device according to claim 14, wherein saidportable terminal device is a cellular phone or a portable informationterminal.
 23. A portable terminal device comprising: an imaging unitthat images an object; a display unit that displays an image produced bysaid imaging unit; a code recognizing unit that recognizes a code ofsaid imaged object; and an imaging procedure switching unit thatswitches the imaging unit to a code mode in response to a useroperation, wherein said imaging unit is disposed on a back side of amain body, and said display unit is disposed on a front side of saidmain body, wherein, after the imaging unit is switched to the code modeby said imaging procedure switching unit, said image is displayed onsaid display unit continuously at a frame rate required by said coderecognizing unit while the display magnification is changed inaccordance with the type of code that can be read, and wherein, oncecode recognition is successfully completed, the result of coderecognition is displayed on said display unit instead of said image. 24.The portable terminal device according to claim 23, wherein saidimagining procedure switching unit changes an image captured area insaid imaging unit depending on the type of code.
 25. The portableterminal device according to claim 23, wherein said imaging procedureswitching unit changes display magnification on said display unitdepending on the type of code.
 26. The portable terminal deviceaccording to claim 23, wherein said type of code includes the differencebetween a one-dimensional code and a two-dimensional code.
 27. Theportable terminal device according to claim 23, wherein said imagingprocedure switching unit changes at least one of gradation property,contrast, brightness, white balance, exposure, focus depth, edgeenhancement, and resolution when the imaging unit is switched to thecode mode.
 28. The portable terminal device according to claim 23,further comprising a notification unit for notifying the result ofrecognition by said code recognizing unit.
 29. The portable terminaldevice according to claim 20, further comprising a timer setting unitthat counts a set time, wherein said notification unit notifies thefailure of recognition upon failure to recognize a code within a timeset by said timer setting unit.
 30. The portable terminal deviceaccording to claim 23, wherein the result of recognition is displayed onsaid display unit after code recognition is completed.
 31. The portableterminal device according to claim 23, wherein said portable terminaldevice is a cellular phone or a portable information terminal.
 32. Amethod of imaging an object using a portable terminal device comprisingthe steps of: producing and displaying an image of the object on adisplay unit; switching the imaged object to a code; increasing themagnification of the image on the display unit; and performing coderecognition continuously while continuously displaying the image on thedisplay unit at a frame rate required for code recognition.
 33. Acomputer readable recording medium recorded with a program for causing aportable terminal device to execute the following steps: producing anddisplaying an image of the object on a display unit; switching the imageto a code; increasing the magnification of the image on the displayunit; and performing code recognition continuously while continuouslydisplaying the image on the display unit at a frame rate required forcode recognition.
 34. A method of imaging an object using a portableterminal device having an imaging unit that images an object, a displayunit that displays an image produced by said imaging unit, a coderecognizing unit that recognizes a code of said image, a coderecognition instructing unit that activates said code recognizing unitin response to a user instruction, an imaging procedure switching unitthat switches the imaging unit between an object mode and a code mode,wherein said imaging unit is disposed on a back side of a main body, andsaid display unit is disposed on a front side of said main body,comprising the steps of: after the imaging unit has been switched to thecode mode by said imaging procedure switching unit and before said coderecognizing unit is activated, displaying said image on said displayunit continuously with the display magnification changed in accordancewith the type of code that can be read, and after said code recognizingunit is activated, performing code recognition continuously anddisplaying said image on said display unit at a frame rate required bysaid code recognizing unit, and, once code recognition is successfullycompleted, displaying the result of code recognition on said displayunit instead of said image.
 35. A computer readable recording mediumrecorded with a program for causing a portable terminal device having animaging unit that images an object, a display unit that displays animage produced by said imaging unit, a code recognizing unit thatrecognizes a code of said image, a code recognition instructing unitthat activates said code recognizing unit in response to a userinstruction, an imaging procedure switching unit that switches theimaging unit from an object mode to a code mode, wherein said imagingunit is disposed on a back side of a main body, and said display unit isdisposed on a front side of said main body to execute the followingsteps: after the imaging unit has been switched to the code mode by saidimaging procedure switching unit and before said code recognizing unitis activated, displaying said image on said display unit continuouslywith the display magnification changed in accordance with the type ofcode that can be read, and after said code recognizing unit isactivated, performing code recognition continuously and displaying saidimage on said display unit at a frame rate required by said coderecognizing unit, and, once code recognition is successfully completed,displaying the result of code recognition on said display unit insteadof said image.
 36. A method of imaging an object using a portableterminal device having an imaging unit that images an object, a displayunit that displays an image produced by said imaging unit, a coderecognizing unit that recognizes a code of said imaged object, animaging procedure switching unit that switches the imaging unit from anobject mode to a code mode in response to a user operation, wherein saidimaging unit is disposed on a back side of a main body, and said displayunit is disposed on a front side of said main body, comprising the stepsof: switching the imaging unit to the code mode; changing the displaymagnification in accordance with the type of code that can be read,displaying the image on said display unit continuously at a frame raterequired by said code recognizing unit; and, once code recognition issuccessfully completed, displaying the result of code recognition onsaid display unit instead of said image.
 37. A computer readablerecording medium recorded with a program for causing a portable terminaldevice having an imaging unit that images an object, a display unit thatdisplays an image produced by said imaging unit, a code recognizing unitthat recognizes a code of said imaged object, an imaging procedureswitching unit that switches the imaging unit from an object mode to acode mode in response to a user operation, wherein said imaging unit isdisposed on a back side of a main body, and said display unit isdisposed on a front side of said main body, to execute the followingsteps: switching the imaging unit to a code mode; changing the displaymagnification in accordance with the type of code that can be read,displaying the image on said display unit continuously at a frame raterequired by said code recognizing unit; and, once code recognition issuccessfully completed, displaying the result of code recognition onsaid display unit instead of said image.
 38. A portable terminal devicecomprising: an imaging unit that images an object; an imaging procedureswitching unit that switches an imaged object to a code; a display unitthat displays an image produced by said imaging unit; a code recognizingunit that recognizes a code of said imaged object; a control unit thatcauses, once the imaged object is switched to a code by said imagingprocedure switching unit, image information to be displayed on saiddisplay unit with an increased magnification, a frame rate control unitthat sets the frame rate of display prior to the start of code readingto be higher than the frame rate after the start of code reading andcontinuously code recognizing.
 39. A portable terminal device accordingto claim 38, wherein the display is performed at a frame rate requiredby the code recognizing unit during code recognizing and continuouslycode recognizing.
 40. The portable terminal device according to claim38, wherein said imaging procedure switching unit changes an imagecaptured area in said imaging unit depending on the type of code. 41.The portable terminal device according to claim 40, wherein said type ofcode includes the difference between a one-dimensional code and atwo-dimensional code.
 42. The portable terminal device according toclaim 38, wherein said imaging procedure switching unit changes displaymagnification of image information on said display unit depending on thetype of code.
 43. The portable terminal device according to claim 38,wherein said imaging procedure switching unit changes at least one ofgradation property, contrast, brightness, white balance, exposure, focusdepth, edge enhancement, and resolution when the imaged object isswitched to code.
 44. The portable terminal device according to claim38, further comprising a notification unit for notifying the result ofrecognition by said code recognizing unit.
 45. The portable terminaldevice according to claim 44, wherein said notification unit notifiesthe result of recognition via sound, voice, or a message displayed onsaid display unit.
 46. The portable terminal device according to claim44, further comprising a timer setting unit that counts a set time,wherein said notification unit notifies the failure of recognition uponfailure to recognize a code within a time set by said timer settingunit.
 47. The portable terminal device according to claim 38, whereinthe result of recognition is displayed on said display unit after coderecognition is completed.
 48. The portable terminal device according toclaim 38, wherein said portable terminal device is a cellular phone or aportable information terminal.
 49. A method of imaging an object using aportable terminal device comprising the steps of: producing anddisplaying an image of the object on a display unit a first frame rate;switching the imaged object to a code; increasing the magnification ofthe image on the display unit; and setting the frame rate of display toa second frame rate lower than the first frame rate.
 50. A computerreadable recording medium recorded with a program for causing a portableterminal device to execute the following steps: producing and displayingan image of the object on a display unit a first frame rate; switchingthe imaged object to a code; increasing the magnification of the imageon the display unit; and setting the frame rate of display to a secondframe rate lower than the first frame rate.